Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

在各种任务和场景中使用多机器人系统的使用越来越兴趣。这种系统的主要吸引力是它们的灵活性，鲁棒性和可扩展性。系统模块化是一个经常被忽视但有希望的功能，它为利用代理专业化提供了可能性，同时还可以实现系统级别的升级。但是，改变代理的能力可以改变最大化系统性能所需的勘探探索示例平衡。在这里，我们研究了群异质性对其探索探索平衡的影响，同时跟踪在对多个移动目标框架的合作多机器人观察下跟踪多个快速移动的回避目标。为此，我们使用分散的搜索和跟踪策略，并具有可调节水平的探索和剥削水平。通过间接调整平衡，我们首先确认这两个关键的竞争动作之间存在最佳平衡。接下来，通过用更快的速度替换较慢的移动剂，我们表明该系统表现出了性能的改进，而无需对原始策略进行任何修改。此外，由于更快的代理商进行了额外的剥削量，我们证明，可以通过降低代理的连接水平来进一步改善异质系统的性能，从而有利于探索性动作的行为。此外，在研究蜂群剂的密度的影响时，我们表明，加快代理的添加可以抵消代理数量的减少，同时保持跟踪性能的水平。最后，我们探索使用差异化策略来利用群体的异质性质的挑战。

图形神经网络（GNN）在许多基于图的任务中表现出强大的表示能力。具体而言，由于其简单性和性能优势，GNN（例如APPNP）的解耦结构变得流行。但是，这些GNN的端到端培训使它们在计算和记忆消耗方面效率低下。为了应对这些局限性，在这项工作中，我们为图形神经网络提供了交替的优化框架，不需要端到端培训。在不同设置下进行的广泛实验表明，所提出的算法的性能与现有的最新算法相当，但具有更好的计算和记忆效率。此外，我们表明我们的框架可以利用优势来增强现有的脱钩GNN。

本文介绍了多传感器用户界面的开发，以促进电弧焊接任务的指导。获得手眼协调能力的传统方法通常是通过一对一的指导来进行的，学员必须戴着保护头盔并进行多项测试。这种方法效率低下，因为电弧从电弧发出的有害光阻止了对过程的密切监测。从业者只能观察到一个小的亮点。为了解决这些问题，最近的培训方法利用虚拟现实来安全地模拟该过程并可视化工件的几何形状。但是，这些类型的仿真平台的合成性质降低了它们的有效性，因为它们无法构成与环境的实际焊接相互作用，从而阻碍了受训者的学习过程。为了为用户提供真正的焊接体验，我们开发了一个新的多传感器扩展现实平台，用于弧焊接训练。我们的系统由：（1）HDR摄像头，实时监视真实的焊接位； （2）深度传感器，捕获场景的3D几何形状； （3）头部安装的VR显示屏，可以安全地可视化过程。我们的创新平台为用户提供了“机器人培训师”，接缝几何形状的虚拟提示，自动点跟踪和性能得分。为了验证平台的可行性，我们通过几项焊接培训任务进行了广泛的实验。我们表明，与传统的培训实践和最近的虚拟现实方法相比，我们的自动多传感器方法在准确性，学习曲线和有效性方面取得了更好的性能。

Selecting the number of topics in LDA models is considered to be a difficult task, for which alternative approaches have been proposed. The performance of the recently developed singular Bayesian information criterion (sBIC) is evaluated and compared to the performance of alternative model selection criteria. The sBIC is a generalization of the standard BIC that can be implemented to singular statistical models. The comparison is based on Monte Carlo simulations and carried out for several alternative settings, varying with respect to the number of topics, the number of documents and the size of documents in the corpora. Performance is measured using different criteria which take into account the correct number of topics, but also whether the relevant topics from the DGPs are identified. Practical recommendations for LDA model selection in applications are derived.

Applying deep learning concepts from image detection and graph theory has greatly advanced protein-ligand binding affinity prediction, a challenge with enormous ramifications for both drug discovery and protein engineering. We build upon these advances by designing a novel deep learning architecture consisting of a 3-dimensional convolutional neural network utilizing channel-wise attention and two graph convolutional networks utilizing attention-based aggregation of node features. HAC-Net (Hybrid Attention-Based Convolutional Neural Network) obtains state-of-the-art results on the PDBbind v.2016 core set, the most widely recognized benchmark in the field. We extensively assess the generalizability of our model using multiple train-test splits, each of which maximizes differences between either protein structures, protein sequences, or ligand extended-connectivity fingerprints. Furthermore, we perform 10-fold cross-validation with a similarity cutoff between SMILES strings of ligands in the training and test sets, and also evaluate the performance of HAC-Net on lower-quality data. We envision that this model can be extended to a broad range of supervised learning problems related to structure-based biomolecular property prediction. All of our software is available as open source at https://github.com/gregory-kyro/HAC-Net/.

Counterfactual explanation is a common class of methods to make local explanations of machine learning decisions. For a given instance, these methods aim to find the smallest modification of feature values that changes the predicted decision made by a machine learning model. One of the challenges of counterfactual explanation is the efficient generation of realistic counterfactuals. To address this challenge, we propose VCNet-Variational Counter Net-a model architecture that combines a predictor and a counterfactual generator that are jointly trained, for regression or classification tasks. VCNet is able to both generate predictions, and to generate counterfactual explanations without having to solve another minimisation problem. Our contribution is the generation of counterfactuals that are close to the distribution of the predicted class. This is done by learning a variational autoencoder conditionally to the output of the predictor in a join-training fashion. We present an empirical evaluation on tabular datasets and across several interpretability metrics. The results are competitive with the state-of-the-art method.

Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge. This paper aims to understand LMs' strengths and limitations in memorizing factual knowledge, by conducting large-scale knowledge probing experiments of 10 models and 4 augmentation methods on PopQA, our new open-domain QA dataset with 14k questions. We find that LMs struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in the tail. We then show that retrieval-augmented LMs largely outperform orders of magnitude larger LMs, while unassisted LMs remain competitive in questions about high-popularity entities. Based on those findings, we devise a simple, yet effective, method for powerful and efficient retrieval-augmented LMs, which retrieves non-parametric memories only when necessary. Experimental results show that this significantly improves models' performance while reducing the inference costs.

We introduce the Conditional Independence Regression CovariancE (CIRCE), a measure of conditional independence for multivariate continuous-valued variables. CIRCE applies as a regularizer in settings where we wish to learn neural features $\varphi(X)$ of data $X$ to estimate a target $Y$, while being conditionally independent of a distractor $Z$ given $Y$. Both $Z$ and $Y$ are assumed to be continuous-valued but relatively low dimensional, whereas $X$ and its features may be complex and high dimensional. Relevant settings include domain-invariant learning, fairness, and causal learning. The procedure requires just a single ridge regression from $Y$ to kernelized features of $Z$, which can be done in advance. It is then only necessary to enforce independence of $\varphi(X)$ from residuals of this regression, which is possible with attractive estimation properties and consistency guarantees. By contrast, earlier measures of conditional feature dependence require multiple regressions for each step of feature learning, resulting in more severe bias and variance, and greater computational cost. When sufficiently rich features are used, we establish that CIRCE is zero if and only if $\varphi(X) \perp \!\!\! \perp Z \mid Y$. In experiments, we show superior performance to previous methods on challenging benchmarks, including learning conditionally invariant image features.

This paper describes Waymo's Collision Avoidance Testing (CAT) methodology: a scenario-based testing method that evaluates the safety of the Waymo Driver Automated Driving Systems' (ADS) intended functionality in conflict situations initiated by other road users that require urgent evasive maneuvers. Because SAE Level 4 ADS are responsible for the dynamic driving task (DDT), when engaged, without immediate human intervention, evaluating a Level 4 ADS using scenario-based testing is difficult due to the potentially infinite number of operational scenarios in which hazardous situations may unfold. To that end, in this paper we first describe the safety test objectives for the CAT methodology, including the collision and serious injury metrics and the reference behavior model representing a non-impaired eyes on conflict human driver used to form an acceptance criterion. Afterward, we introduce the process for identifying potentially hazardous situations from a combination of human data, ADS testing data, and expert knowledge about the product design and associated Operational Design Domain (ODD). The test allocation and execution strategy is presented next, which exclusively utilize simulations constructed from sensor data collected on a test track, real-world driving, or from simulated sensor data. The paper concludes with the presentation of results from applying CAT to the fully autonomous ride-hailing service that Waymo operates in San Francisco, California and Phoenix, Arizona. The iterative nature of scenario identification, combined with over ten years of experience of on-road testing, results in a scenario database that converges to a representative set of responder role scenarios for a given ODD. Using Waymo's virtual test platform, which is calibrated to data collected as part of many years of ADS development, the CAT methodology provides a robust and scalable safety evaluation.